Advertising on Internet websites has become a popular method of advertising. In addition, website publishers can generate additional income by allowing advertisements to appear with their web pages. Currently, when a web browser is used to display and interact with text on a web page, the Internet service provider may connect with the website, which may request an advertisement from an ad server. The ad server, which stores advertisements sent from advertising agencies, may select an advertisement and an image to present to the computer user.
To target specific customer types, advertisers may specify a uniform resource locator (URL) on which to post an advertisement, or may just allow their advertisement to be posted on any webpage. A web crawler (a.k.a. web spider) is a type of software program that methodically browses the Internet to gather information. The web crawler may obtain data that can be indexed later by a search engine to speed up web searches.